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This  thesis  investigates  relationships  between  extensions 
in  technology  and  both  cost  and  schedule  slippages  in  the 
development  and  production  phases  of  weapons  system  acquisi¬ 
tion.  The  primary  objective  is  to  determine  if  the  amount  of 
technology  embodied  in  a  given  weapons  system  can  be  employed 
to  predict  a  Department  of  Defense  (DOD)  contractor's  perfor¬ 
mance  in  meeting  cost  and  schedule  targets.  The  analysis  used 
a  sample  of  15  U.S.  military  tactical  air-launched  munitions 
systems.  It  begins  with  a  review  of  the  literature  regarding 
technology  measurement  and  its  connection  to  cost  and  schedule 
outcomes.  Next,  measures  of  technological  progress  are 
developed  and  displayed.  Third,  the  process  of  creating  cost 
growth  and  schedule  slippage  measures  are  discussed.  The 
relationships  between  technological  complexity  and  cost  and 
schedule  outcomes  are  then  empirically  tested.  Major  findings 
indicate  that  measures  of  extension  in  technology  are  worth¬ 
while  for  explaining  production  and  total  program  cost  growth. 


iv 


4  r  _  _  _ _ 

'  ■  r  1  O’jt  X  \  wij . 

I 

j 

i 


DIM 


TABLE  OF  CONTENTS 


I.  INTRODUCTION  AND  LITERATURE  REVIEW  -  1 

A.  BACKGROUND -  1 

B.  PURPOSE -  1 

C.  LITERATURE  REVIEW  -  2 

D.  MEASURING  EXTENSIONS  OF  TECHNOLOGY  -  8 

E.  BENEFITS -  10 

F.  OBJECTIVES -  11 

G.  SUBSEQUENT  CHAPTERS  -  11 

II.  SAMPLE,  DATA,  AND  HYPOTHESIS -  14 

A.  SAMPLE -  14 

B.  TECHNOLOGY  DATA -  16 

C.  COST  AND  SCHEDULE  DATA -  18 

D.  HYPOTHESES -  21 

E.  SUMMARY -  24 

III.  TECHNOLOGY,  COST,  AND  SCHEDULE  MEASUREMENT  -  26 

A.  CREATING  TECHNOLOGY  MEASURES  -  26 

B.  COST  AND  SCHEDULE  MEASURES -  3  3 

C.  SUMMARY -  41 

IV.  STATISTICAL  ANALYSIS  AND  TESTS  -  42 

A.  ANALYSIS  OF  VARIABLES -  4  3 

B.  TESTS  OF  HYPOTHESES -  47 

C.  OTHER  CONSIDERATIONS  -  54 

D.  RESULTS -  57 


V 


V.  SUMMARY  AND  CONCLUSIONS -  62 

A.  SUMMARY -  62 

B.  CONCLUSIONS -  64 

APPENDIX  A;  MISSILE  DESIGNATION  SYSTEM  -  66 

APPENDIX  B:  PROPERTIES,  CHARACTERISTICS,  AND 
PERFORMANCE  MEASURES  DESCRIBING 
TACTICAL  AIR-LAUNCHED  MUNITIONS 

TECHNOLOGY -  68 

APPENDIX  C:  YEAR  OF  TECHNOLOGY  REGRESSION  ANALYSIS  —  70 

APPENDIX  D;  CONTROL  FACTORS  THAT  AFFECT  COST 

GROWTH -  73 

APPENDIX  E:  COST  GROWTH  OUTCOME  CONTROL  MEASURES  -  74 

LIST  OF  REFERENCES -  75 

INITIAL  DISTRIBUTION  LIST  -  78 


VI 


LIST  OF  TABLES 


II-l  SAMPLE  TACTICAL  AIR-LAUNCHED  MUNITIONS 

SYSTEMS -  15 

II-2  VARIABLES  DESCRIBING  TACTICAL  AIR-LAUNCHED 

MUNITIONS  TECHNOLOGY  -  19 

II- 3  PROGRAM  OUTCOMES -  21 

III- l  VARIABLE  REDUCTION;  UNIVARIATE  REGRESSION 

RESULTS,  VERSION  A -  29 

III-2  VARIABLE  REDUCTION:  STEPWISE  REGRESSION, 

VERSION  B -  3  0 

III-3  VARIABLE  REDUCTION:  JUDGMENTAL  METHOD, 

VERSION  C -  32 

III-4  YEAR  OF  TECHNOLOGY  PLOT,  VERSION  A -  34 

III-5  YEAR  OF  TECHNOLOGY  PLOT,  VERSION  B -  35 

I II -6  YEAR  OF  TECHNOLOGY  PLOT,  VERSION  C -  36 

III-7  TECHNOLOGY  MEASURES  -  37 

III- 8  OUTCOME  MEASURES  -  40 

IV-  1  TECHNOLOGY  AND  OUTCOME  MEASURES  -  47 

IV-2  DEVELOPMENT  COST  GROWTH,  MULTIVARIATE 

REGRESSIONS -  48 

IV- 3  PRODUCTION  COST  GROWTH,  MULTIVARIATE 

REGRESSIONS -  49 

IV-4  TOTAL  PROGRAM  COST  GROWTH,  MULTIVARIATE 

REGRESSIONS -  50 

IV-5  DEVELOPMENT  SCHEDULE  GROWTH, 

MULTIVARIATE  REGRESSIONS  -  51 

IV-6  PRODUCTION  SCHEDULE  GROWTH,  MULTIVARIATE 

REGRESSIONS -  52 


Vll 


IV-7  DEVELOPMENT  COST  GROWTH  WITH  CONTROLS, 

MULTIVARIATE  REGRESSIONS  -  58 

IV-8  PRODUCTION  COST  GROWTH  WITH  CONTROLS, 

MULTIVARIATE  REGRESSIONS  -  59 

IV-9  TOTAL  PROGRAM  COST  GROWTH  WITH  CONTROLS, 

MULTIVARIATE  REGRESSIONS  -  60 


viii 


I.  INTRODUCTION  AND  LITERATURE  REVIEW 


A.  BACKGROUND 

A  number  of  prior  studies  have  used  data  describing 
measures  of  performance  and  other  properties  and 
characteristics  as  indicators  of  technology  embodied  in  a 
system.  These  studies  then  endeavored  to  relate  these 
measures  to  contractor  cost  and  schedule  performance.  This 
thesis  is  a  follow-on  to  these  works.  This  chapter  begins 
with  a  literature  review  of  the  techniques  currently  available 
involving  technology  measurement  and  its  relationship  with 
cost  and  schedule  outcomes.  It  will  then  describe  how  the 
concepts  in  those  works  can  be  used  as  the  groundwork  for  the 
analysis  conducted  in  this  thesis  and  discuss  the  basic 
methodology  to  be  used.  This  first  chapter  will  conclude  with 
a  preview  of  subsequent  chapters. 

B.  PURPOSE 

The  intent  of  this  thesis  is  to  determine  what,  if  any, 
relationship  exists  between  the  state  of  the  art  in  technology 
and  advances  in  technology  embodied  in  weapons  systems  and 
contractor  cost  and  schedule  performance  during  acquisition  of 
these  systems.  It  will  attempt  to  analyze  the  particular 
relationships  between  technological  sophistication  and 
advances  in  technology  of  a  set  of  U.S.  military  air-launched 
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tactical  munitions  systems  and  a  Department  of  Defense  (DOD) 
contractor's  ability  to  achieve  control  over  cost  growth  and 
schedule  slippages  in  both  the  development  and  production 
phases . 

C.  LITERATURE  REVIEW 

In  this  section,  several  key  studies  are  reviewed  that 
apply  to  the  objective  of  this  thesis  and  to  the  methodology 
to  be  used  in  the  analysis.  That  is,  the  following  studies 
examine  methods  to  create  technology  measures  and/or 
procedures  to  relate  the  level  of  technology  in  a  system  to 
cost  performance.  Each  of  these  studies  has  shown  that 
variables  reflecting  system  performance,  properties,  and 
characteristics  can  be  combined  to  measure  the  amount  of 
technology  embodied  in  a  system.  These  technology  describing 
variables  can  also  be  used  to  measure  advances  in  the 
technology  of  upgraded/updated  systems.  These  studies  require 
the  accumulation  of  numerous  technology  describing  variables 
that  distinctly  indicate  a  system's  performance,  properties, 
and  characteristics.  Each  approach  then  combines  variables 
into  a  single  measure  that  reflects  the  quantity  of  technology 
in  a  given  system.  Some  of  the  works  have  then  attempted  to 
relate  the  level  of  technology  to  system  development  and/or 
production  costs.  A  common  theme  is  an  understanding  that  the 
cost  of  extending  technology  relates  to  the  magnitude  of  the 
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extension.^  [Refs.  1,2,3]  The  analysis  in  this  thesis  builds 
upon  the  work  of  the  authors  reviewed.  The  following  reviews 
are  presented  in  chronological  order. 

1 .  Alexander  and  Nelson.  1972 

Alexander  and  Nelson  [Ref.  6]  use  a  linear  regression 
method  to  analyze  their  data.  A  noteworthy  point,  applicable 
to  the  discussion  in  this  thesis,  lies  in  their  discussion  of 
the  use  of  a  graphically  displayed,  regression  trend  line  to 
identify  the  average  or  expected  date  of  technology  of  systems 
over  time.  This  allowed  them  to  display  the  level  of 
technology  embodied  in  a  given  system.  Points  representing 
individual  systems  that  were  plotted  above  the  trend  line 
indicated  that  the  technology  in  that  system  was  "ahead  of  its 
time."  Similarly,  points  below  the  line  indicated  that  the 
systems  were  "late"  or  "conservative"  developments. 

2 .  Dodson.  1977 

In  this  study,  Dodson  [Ref.  7]  used  three  technology 
indicators.  He  ran  a  multivariate  regression  with  the  year  of 
development  as  the  dependent  variable.  The  regression 
equation  was  in  the  following  form: 

Ye  =  ko  +  kjXi  +  k2X2  + 


^Important  research  by  such  authors  as  Dodson  &  Graver 
[Ref.  4]  and  Dodson  [Ref.  5]  are  not  reviewed  here.  These  two 
studies,  among  others,  made  use  of  ellipsoidal  hypersurfaces 
to  represent  levels  of  technology.  They  represent  important 
contributions  to  the  field  of  technology  measurement,  but  are 
not  directly  relevant  to  the  focus  of  this  analysis. 
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where : 


Ye  =  The  Year  of  Technology  (defined  as  the  year 
that  the  system  should  have  been  introduced 
given  the  technology  embodied  in  the  system) 

ki  =  The  regression  coefficients 

Xi  =  The  technology  indicators. 

He  calculated  Y^  for  each  system  in  the  sample  and  postulated 
that  if  the  actual  year  of  development  was  less  than  Y^,  then 
the  system  was  "ahead  of  its  time."  Similarly,  if  the  actual 
year  of  development  was  greater  than  Y^,  then  the  system  was 
technologically  "behind  the  times"  when  it  was  introduced. 
Therefore,  Y^  -  YActuai  was  a  measure  of  the  technological 
advance  represented  by  each  system  in  the  sample. 

In  an  effort  to  associate  costs  with  technology, 
Dodson  next  conducted  another  regression  with  cost  as  the 
dependent  variable  and  technology  advance  (Y^  -  Y^ctuai)  as  an 
independent  variable.  He  estimated  both  Research  & 

Development  (R&D)  costs  and  procurement  costs  in  separate 
regressions.  Procurement  cost  was  approximated  from  a  cost 
estimating  relationship  (CER)  incorporating  the  Year  of 
Technology.  This  was  necessary  to  compensate  for  the  changes 
in  production  technology  over  time.  Dodson  reasoned  that  an 
advanced  design  would  be  relatively  more  expensive  to  produce 
with  oldar  manufacturing  technology.  At  the  same  time, 
advanced  manufacturing  technology  should  result  in  a  drop  in 
procurement  cost.  He  found  that  his  estimates  for  R&D  costs 
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were  not  completely  satisfactory,  however.  This  was  largely 
due,  he  explained,  to  certain  unidentified  costs  of  R&D  that 
were  not  included  in  the  regression  model.  Dodson  had 
somewhat  better  success  with  procurement  costs. 

3 .  Gordon  and  Munson.  1981^ 

This  paper  examined  technology  extension  and  its 
measurement  using  two  types  of  equations  [Ref.  8]: 

S  =  KiVi  +  K2V2  +  .  .  .  +  K„V„ 


and 


S  =  Vi[K2V2  +  K3V3  +  .  .  .  +  K„V„] 

where : 

S  =  State  of  the  Art  (the  current  extent  of 
technology) 

Ki  =  A  judgmental  or  statistical  weight 

Vi  =  The  value  of  the  ith  technology  describing 

variable . 


The  first  is  a  simple  linear  equation  combining  weighted 
characteristics.  The  second  equation  is  a  multiplicative  foirm 
that  is  intended  to  be  used  when  one  variable  must  be  present. 

An  important  new  contribution  made  by  Gordon  and 
Munson  in  this  paper  was  to  propose  factor  analysis  as  a  means 
to  collect  a  large  number  of  technology  describing  variables 


^Discussion  adapted  from  review  in  Greer  [Ref.  l:pp.  11- 

13]  . 
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into  groups  that  have  similar  relation  to  or  influence  on  the 
state  of  the  art  in  technology.  The  advantage  of  factor 
analysis  is  that  it  allows  for  a  large  number  of  variables  to 
be  in  iorporated  into  the  model  by  aggregating  several 
correlated  variables  into  a  few,  more  manageable,  composite 
measures  called  factors.  The  factor  scores  can  then  be 
employed  for  further  analysis.  Factor  analysis  represents  yet 
another  tool  for  relating  technology  and  cost  measures. 

4 .  Knight.  1985 

Two  important  contributions  were  made  in  this  study 
[Ref.  9].  First,  Knight  distinguished  between  functional  and 
structural  technology  measures  and  related  the  two.  He 
defined  a  functional  measure  of  technology  as:  "The 
capability  of  each  system  to  perform  its  intended  tasks." 
[Ref.  9:p.l07]  Knight  also  studied  functional  technology 
measures  over  time.  This  study  provided  guidelines  for  data 
collection. 

5 .  Greer.  1988 

These  previous  studies  examined  techniques  to  define 
extensions  in  technology,  usually  with  the  use  of  regression 
analysis  as  a  common  thread.  Technology  describing  variables 
were  typically  combined  to  produce  single  measures  represent¬ 
ing  the  amount  of  technology  in  a  system.  The  "Year  of 
Technology"  was  usually  employed  as  the  dependent  variable  in 
the  regression  equation.  Several  studies  then  attempted  to 
relate  the  scale  of  technology  to  system  costs. 


Building  on  these  works,  Greer  [Ref.  1]  undertook  to 
develop  a  cost  estimating  model,  using  technolo-^y  measurement 
methods,  that  could  both  predict  costs  and  be  useful  for  cost 
control  purposes.  Using  technology  measures,  he  first 
developed  a  time  regression  to  predict  the  amount  of  time 
required  for  development  of  a  system.  This  predicted  time  was 
then  used  in  a  cost  regression  calculating  an  "ex  ante"  cost 
estimate  for  each  system. 

Next,  the  actual  time  for  the  project  was  compared  to 
the  predicted  time  to  calculate  a  residual  (similar  to 
Dodson's  Ye  -  ^Actual  Computation).  The  cost  regression  was  run 
again  to  produce  a  new  cost  estimate  based  on  the  residual 
project  time.  The  difference  between  the  ex  ante  cost 
estimate  and  the  cost  estimate  based  on  actual  time  was  called 
the  "Variance  Due  to  Time."  This  represents  that  portion  of 
variance  explained  by  time  delays.  Greer  also  compared  actual 
cost  to  the  cost  estimate  based  on  actual  time  to  ascertain  a 
"Cost  Control  Variance."  This  variance  revealed  the  quality 
of  a  project's  cost  control. 

In  developing  the  two  variance  measures,  Greer  hoped 
to  explain  the  causes  of  cost  variance  for  each  system  he 
analyzed.  The  results  indicated  whether  observed  variances 
were  due  to  time  or  to  cost  control.  In  order  to  control 
costs,  he  concluded,  it  was  essential  to  explain  the  variance 
between  predicted  and  actual  costs. 
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Greer  also  noted  that  development  cost  is  not  a  smooth 
function  of  time.  "If  a  program  drags  on  beyond  its  intended 
completion  date,  it  becomes  relatively  more  costly  to  compress 
the  required  accomplishment  into  an  increasingly  abbreviated 
time  horizon."  [Ref.  l:p.  vi] 

6 .  Moses.  1989 

In  this  study,  Moses  [Ref.  3]  revised  and  extended 
Greer's  work.  He  employed  the  "Year  of  Technology"  approach 
(the  focus  in  this  thesis)  to  technology  measurement  as  did 
Alexander  &  Nelson  and  Dodson  rather  than  Greer's  use  of  the 
ellipsoid  method. 

Like  Greer,  Moses  addressed  the  relationship  between 
technology  and  development  costs  and  created  methods  for 
determining  development  cost  variances.  In  an  important  new 
thrust,  the  report  extends  the  analysis  to  examine  production 
costs  as  well  as  development  costs.  Moses  used  data  for  18 
satellite  systems  to  empirically  test  this  approach  to 
technology  measurement  and  cost  prediction  and  control. 

D.  MEASURING  EXTENSIONS  OF  TECHNOLOGY 

The  studies  reviewed  here  focus  on  methods  to  measure 
technology.  One  commonly  used  method  is  the  "Year  of 
Technology"  Regression  approach.  This  procedure  uses  multiple 
regression  to  relate  several  technology  describing  variables 
and  time  in  the  following  general  form: 
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Y  =  Ko  +  KiXi  +  K2X2  +  .  .  .  +  K„X„  +  e 


where : 

Y  =  The  actual  year  that  the  system  was  produced 
or  became  operational 

Ki  =  Regression  coefficients 

Xi  =  Technology  describing  variables 

e  =  Residual. 


Values  predicted  from  the  regression  equation  for  a  given 
system  represent  the  "Year  of  Technology"  for  that  system.  As 
Dodson  noted,  if  the  actual  year  that  the  system  was  produced 
or  became  operational  is  less  than  the  calculated  year  of 
technology,  then  the  system  represents  an  advancement  in 
technology. 

According  to  Moses,  the  essence  of  the  Year  of 
Technology  method  is  to  combine  a  number  of  technology 
variables  into  summary  measures  and  express  them  in  terms  of 
time  (years) .  Using  this  approach,  he  created  three 
technology  measures  for  each  system  in  his  sample:  STAND, 
ADVANCE,  and  REACH.  These  are  briefly  defined  as  follows; 

-  STAND — The  actual  year  that  the  system  becomes 
operational  which  represents,  in  general,  where 

technology  currently  "stands."  Stand  equals  "Y"  in 
Dodson's  equation  Y^  -  Y.  For  this  thesis,  STAND  is  equal 
to  Full-Scale  Development  (FSD)  Start  Date. 

ADVANCE — The  extension  of  technology  beyond  the  current 
state  of  the  art.  This  measure  is  the  deviation  from  the 
trend  line  discussed  by  Alexander  &  Nelson.  In  Dodson's 
equation,  ADVANCE  =  Y^  -  Y. 
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REACH — A  measure  of  the  level  of  technology  embodied  in 
a  system  equal  to  Dodson's  "Ye-" 

This  thesis  will  use  STAND,  ADVANCE,  and  REACH  measures  to 

summarize  the  technology  associated  with  a  particular  weapon 

system.  These  measures  will  then  be  used  in  an  attempt  to 

explain  cost  and  schedule  outcomes. 

A  related  work  by  Lienhard  [Ref.  10]^  supports  the 
"Year  of  Technology"  approach.  In  examining  several 
technologies  over  time,  Lienhard  studied  the  rate  that 
technology  improves  and  whether  this  rate  changes  over  time. 
He  noted  that,  once  established,  the  rate  of  improvement  of  a 
given  technology  does  not  change.  If  so,  attempts  to 
accelerate  advances  in  technology  "ahead  of  their  time"  could 
be  extremely  costly.  Therefore,  the  Year  of  Technology 
approach  appears  sound  and  it  is  this  approach  that  will  be 
used  in  this  analysis. 

E.  BENEFITS 

Building  on  the  techniques  established  in  the  literature, 
this  analysis  will  examine  a  sample  of  several  U.S.  military 
tactical  air-launched  munitions  systems.  It  will  contribute 
to  the  existing  body  of  knowledge  by  empirically  analyzing 
these  systems  using  the  methods  described  earlier.  It  seeks 
to  estimate  and  explain  a  government  contractor's  tendency 
towards  cost  growth  and  schedule  growth  resulting  from 

^Discussion  adapted  from  Greer  [Ref.  l:p.  38]. 
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advances  in  and  extensions  to  these  high-technology  weapons 
systems . 


F.  OBJECTIVES 

The  objective  of  this  thesis  is  to  determine  if  there  is 

a  relationship  between  the  state  of  technology  in  air-launched 

tactical  munitions  and  a  contractor's  performance  in  meeting 

cost  and  schedule  targets.  Using  the  techniques  reviewed  in 

the  literature,  the  following  will  be  undertaken: 

A  statement  and  discussion  of  the  hypothesis  that  a 
relationship  may  exist  between  system  technology  and 
contractor  cost  and  schedule  performance. 

Development  of  indicators  of  technology  for  -.ir-launched 
tactical  munitions  that  can  be  employed  to  assess  a 
contractor's  ability  to  meet  cost  and  schedule  targets. 

Using  a  "Year  of  Technology"  analysis  on  tactical  air- 
launched  munitions  data,  construct  summary  technology 
measures. 

-  Examine,  discuss,  and  display  procedures  for  creating 
measures  of  cost  and  schedule  growth. 

Perform  a  statistical  analysis  of  the  hypothesis  and 
review  results  and  findings. 


G .  SUBSEQUENT  CHAPTERS 
1 .  Chapter  II 

This  chapter  will  begin  with  a  description  of  the 
origin  of  the  sample  and  the  data  sources  for  technology 
measures.  Following  techniques  demonstrated  by  Moses  [Ref. 
3],  technology  measures  for  tactical  air-launched  munitions 
will  be  created  using  indicators  of: 


Speed 

Range 

Guidance  Systems 
Sensors 

System  Reliability. 

Next,  data  sources  for  cost  and  schedule  outcomes  will  be 
discussed.  The  chapter  will  conclude  by  defining  the 
hypothesis.  The  hypothesis  is  based  on  how  three  measures  of 
technology:  stand,  advance,  and  reach,  relate  to  cost  growth 

and  schedule  slippages. 

2 .  Chapter  III 

This  chapter  will  examine  the  process  of  creating 
technology  measures  and  will  include  the  "Year  of  Technology” 
regression  analysis  and  a  display  of  measures.  It  will  also 
examine  the  process  of  developing  cost  and  schedule  measures 
through  a  discussion  and  review  of  procedures  and  data  in  the 
study  by  Tyson  et  al.  [Ref.  11].''  It  will  address  controls 
and  contain  a  display  of  measures. 

3 .  Chapter  IV 

Chapter  IV  will  provide  the  statistical  analysis  of 
the  hypothesis  introduced  in  Chapter  II.  It  will  test  the 
relationship  of  cost  and  schedule  growth,  during  both 
development  and  production  phases  (as  the  dependent 


'’For  brevity's  sake,  this  1989  study  by  Tyson,  Nelson,  Om, 
and  Palmer  is  hereinafter  referred  to  simply  as  the  "Tyson 
study. " 
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variables) ,  and  the  level  of  system  technology  (reach,  stand, 
and  advance  as  the  independent  variables)  in  a  regression 
analysis.  Included  will  be  a  discussion  and  display  of 
results. 

4 .  Chapter  V 

This  final  chapter  will  summarize  the  key  aspects  of 
the  analysis  and  present  the  conclusions  of  the  study. 
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II.  SAMPLE.  DATA.  AND  HYPOTHESIS 

This  chapter  is  organized  into  four  sections.  The  first 
presents  the  tactical  air-launched  munitions  systems  to  be 
used  as  the  sample  in  this  thesis.  The  second  and  third 
sections  discuss  the  data  and  its  sources.  Technology  data 
are  addressed  first  and  are  followed  by  cost  and  schedule 
information.  With  an  understanding  of  the  sample  and  data  to 
be  used  in  this  analysis,  the  final  section  of  this  chapter 
will  introduce  the  basic  hypotheses. 

A.  SAMPLE 

The  sample  consists  of  15  tactical  air-launched  munitions 
systems  used  in  the  Tyson  study  [Ref.  ll;see  p.  III-2]. 
Tyson's  original  sample  consisted  of  16  systems.  One,  the 
AGM-53A  Condor,  retired  circa  1979,  was  omitted  in  this  thesis 
due  to  a  lack  of  available  technology  measures.  Table  II-l  is 
a  list  of  the  sample  systems  and  their  Full-Scale  Development 
(FSD)  start  dates. 

These  15  systems  represent  three  distinct  subsets  of 
missile  types  with  unique  mission  requirements:  (1)  Air-to- 
air  (anti-air) ,  (2)  Air-to-surface  (tactical  air-launched) , 
and  (3)  Air-to-Surface  (helicopter  launched)  .  As  will  be  seen 
in  Chapter  III,  these  mission  differences  will  have  an  impact 
on  the  variable  selection  process.  Tyson's  data  identifies 


Table  II-l 


SAMPLE  TACTICAL  AIR-LAUNCHED  MUNITIONS  SYSTEMS 


OBS 

Name 

Desionation 

FSDSD 

1 

Sparrow 

AIM-7E 

1960 

2 

Sparrow 

AIM-7F 

1966 

3 

Sparrow 

AIM-7M 

1978 

4 

Sidewinder 

AIM-9L 

1971 

5 

Sidewinder 

AIM-9M 

1976 

6 

Phoenix 

AIM-54A 

1962 

7 

Phoenix 

AIM-54C 

1977 

8 

HARM 

AGM-88A 

1978 

9 

Harpoon 

AGM-84A 

1973 

10 

Maverick 

AGM-65A/B 

1968 

11 

Maverick 

AGM-65D/G 

1976 

12 

AMRAAM 

AIM-120A 

1982 

13 

Hellf ire 

AGM-114A 

1976 

14 

TOW 

BGM-71A/B 

1963 

15 

TOW2 

BGM-71D 

1978 

the  missile  systems  by  mission,  design,  and  series  using 
standard  missile  system  designation.  A  brief  explanation  of 
the  missile  designation  structure  is  provided  in  Appendix  A. 
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There  are  a  variety  of  air-launched  munitions  that  are  not 
missiles — gravity  bombs  are  one  such  example.  However,  this 
sample  consists  solely  of  missiles.  Therefore,  for  the 
purposes  of  this  thesis,  the  terms  missile  and  munitions  are 
used  interchangeably. 

The  sample  is  spread  over  22  years  when  grouped  by  FSD 
start  date.  All  of  the  programs  are  still  in  production  or  in 
service  with  the  U.S.  Navy,  Army,  Air  Force,  and/or  Marines  or 
in  foreign  service.  Where  a  successor  system  of  a  particular 
missile  resulted  in  a  design  that  was  technologically 
indistinguishable  from  its  predecessor,  Tyson  combined  systems 
into  a  single  program.  Thus,  for  example,  the  AGM-65A  and 
AGM-65B  versions  become  the  AGM-65A/B. 

B .  TECHNOLOGY  DATA 

Data,  consisting  of  numerical  values  for  the  technology 
measures  to  be  considered  in  the  analysis,  were  obtained  from 
a  variety  of  sources  [Refs.  12-17  and  additional  classified 
sources].^  Prior  to  actual  data  collection,  it  was  necessary 
to  select  the  variables  to  be  considered  in  this  analysis. 

The  selection  of  measures  describing  the  amount  of 
technology  embodied  in  each  system  required  certain  pre¬ 
considerations.  According  to  Dodson  and  Graver,  technology 

^The  author  is  indebted  to  the  Naval  Postgraduate  School 
and,  in  particular,  Dr.  Ball  of  the  Department  of  Aeronautical 
Engineering  for  help  in  developing  technology  measures. 
Invaluable  assistance  was  also  provided  by  MAJ  Paul  F. 
O'Sullivan,  Jr.,  USA  and  MAJ  Michael  Staggs,  USA. 
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describing  properties  must  be,  at  least  to  some  extent,  a 
consequence  of  engineering  development  rather  than  a  by¬ 
product  of  that  process.  The  variables  selected  should  be 
design  goals  of  system  development.  Ideally,  these  variables 
should  be  arranged  so  that  increasing  values  display  greater 
technological  sophistication.  Additionally,  the  variables 
selected  should  be  those  whose  values  are  determinable  in  the 
early  stages  of  a  system's  life-cycle  so  that  they  may  be  used 
for  predicting  future  cost  and  schedule  performance  [Ref.  4; 
pp.  13-14]. 

Technical  expertise  was  sought  to  develop  the  list  of 
technology  measures.  A  listing  of  candidate  variables 
describing  the  technical  characteristics,  properties,  and 
performance  criteria  of  the  tactical  air-launched  munitions 
systems  was  developed.  This  candidate  list  is  provided  in 
Appendix  B. 

Some  of  these  characteristics,  properties,  and  performance 
measures  were  considered  to  be  simply  by-products  of  design  or 
were  not  stated  in  a  form  that  could  reveal  much  about 
technological  sophistication.  These  measures  were  dropped 
from  further  consideration.  The  list  of  candidates  was 
further  reduced  by  data  availability.  Due  to  the  variety  of 
missions  represented  in  the  sample,  and  due  to  programmatic 
and  record-keeping  differences  among  the  military  services, 
many  of  the  candidate  measures  are  not  used  or  maintained  for 
individual  systems.  The  result  of  the  data  collection  and 
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review  process  was  the  identification  of  14  variables  that 
could  be  used  to  describe  the  technology  embodied  in  the 
missile  systems.  These  variables  are  listed  in  Table  II-2. 

Values  were  obtained  for  each  of  the  14  variables,  with 
one  exception:  MSR  was  not  available  for  the  AIM-120A  AMRAAM, 
the  newest  system  in  the  sample.  The  mean  value  of  the 
Sparrow  (AIM-7  series) — the  system  with  the  closest,  similar 
mission  (intermediate  range  air-to-air) — was  inserted  to 
preclude  a  distortion  of  the  data  to  be  used  in  later  portions 
of  the  analysis.^  With  14  variables  to  describe  15  systems, 
further  variable  reduction  was  necessary.  This  process  will 
be  discussed  in  Chapter  III. 

C.  COST  AND  SCHEDULE  DATA 

This  thesis  employs  cost  and  schedule  data  from  the  Tyson 
study  [Ref.  11].  In  this  work,  Tyson,  et  al.,  explored  trends 
in  cost  and  schedule  outcomes  of  a  variety  of  major 
acquisition  programs.  They  also  sought  to  determine  the 
effectiveness  of  initiatives  to  improve  these  outcomes.  The 
study  examined  cost  growth  and  schedule  slippages  by  equipment 
type  (including  tactical  air-launched  munitions),  by  time 
periods,  by  development  and  production  phases,  and  by 
development  type  (new  starts  or  major  modifications) .  The 


^Since  portions  of  the  variables  or  their  values  are 
classified,  they  will  not  be  displayed  here.  As  will  be  seen, 
only  regression  results  are  presented  which  are  not 
compromisable . 
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TABLE  I I -2 


VARIABLES  DESCRIBING 

TACTICAL  AIR-LAUNCHF  MUNITIONS  TECHNOLOGY 

Range  (RNG)  Variables: 

RNGl  -  Maximum  Missile  Range  (Meters) 

RNG2  -  Minimum  Missile  Range  (Meters) 

Altitude  (ALT)  Variadiles: 

ALTl  -  Maximum  Missile  Altitude  (Meters) 

ALT2  -  Minimum  Missile  Altitude  (Meters) 

Speed  (SPD)  Variables: 

SPDl  -  Missile  Speed  (Mach) 

SPD2  -  Maximum  Platform  Launch  Speed  (Mach) 

SPD3  -  Minimum  Platfomn  Launch  Speed  (Mach) 

Terminal  Guidance  System  (TGS)  Variables: 

TGSl  -  TGS  Redundancy 

Coded:  0  =  Single  System 

1  =  Dual  Systems 

2  =  Multiple  S'"*'*’ems 
TGS2  -  TGS  Reliability 

Coded:  0,  1.  or  2  based  on  expert  assessment 

of  system  reliability  (classified) . 
ECCM  -  Improved  Electronic  Counter  Measures  Defeating 
Capability 
Coded:  0  =  No 

1  =  Yes 

Platform  Sensor  Varicdales: 

IMCC  -  Inertial  Mid-Course  Correction  Capability 
Coded:  0  =  No 

1  =  Yes 

DSP  -  Digital  Signal  Processing  Capability 
Coded:  0  =  No 

1  =  Yes 

Missile  Fuzing  Variable: 

FUZE  -  Fuze  Type 

Coded:  0  or  1  based,  as  above,  on  expert 

assessment  of  system  reliability 
(classified) . 

Missile  System  RelieUaility  (NSR)  Variable: 

MSR  -  Percent  Reliability,  Given  a  Successful  Launch 
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reason  for  interest  in  the  Tyson  study  is  its  collection  of 
cost  and  schedule  histories  for  the  sample  of  tactical  air- 
launched  munitions. 

The  Tyson  study  obtained  cost  and  schedule  information 
primarily  from  Selected  Acquisition  Report  (SAR)  data. 
Supplemental  information  was  obtained  from  Development  Concept 
(DCP)  statistics  and  other  data  from  the  military  services. 
Department  of  Defense  (DOD)  acquisition/program  offices,  and 
defense  industry  sources  [Ref.  11,  p.  III-3]. 

Measures  of  outcome  were  based  on: 

COST  GROWTH — Development,  Production,  ard  Total:  An 

indicator  of  good  program  performance  is  the  ability  of 
the  contractor  to  develop  and  produce  the  system 
according  to  the  cost  target. 

SCHEDULE  SLIPPAGE — Development  and  Production:  As 
above,  an  indicator  of  good  performance  is  the  degree  to 
which  a  system  can  be  developed  and  produced  within 
planned  time  frames. 

The  study  produced  measures  for  five  program  outcomes.  These 
are  listed  in  Table  11-3.^  They  used  the  term  "growth"  in 
cost  and  schedule  since  most  program  changes  reflected 
increases  in  program  cost  or  time.  The  study  found  that,  in 
a  few  cases,  such  as  when  development  or  production  quantities 
were  reduced,  cost  and  schedule  "growth"  was  negative.  The 


^They  also  examined  quantity  changes  which  "give  clues  to 
such  issues  as  reasonableness  of  the  development  plan,  the 
degree  of  production  stability,  and  the  prevalence  of  program 
stretchout."  [Ref.  ll;p.  III-5]  We  restrict  our  discussion  in 
this  thesis  to  cost  and  schedule  outcomes. 
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TABLE  I 1-3 

PROGRAM  OUTCOMES 

OUTCOME 

CODE 

Development  Cost  Growth 

DCG 

Production  Cost  Growth 

PCG 

Total  Program  Cost  Growth 

TPCG 

Development  Schedule  Growth 

DSG 

Production  Schedule  Growth 

PSG 

process  used  for  calculating  program  outcome  measures  will  be 
discussed  in  Chapter  III. 


D.  HYPOTHESES 

This  section  discusses  the  proposed  associations  between 
a  government  contractor's  performance  in  meeting  cost  targets 
and  planned  schedules  and  the  technological  complexity 
embodied  in  a  weapons  system.  The  purpose  of  the  study  is  to 
determine  if  a  relationship  exists  between  technology 
extensions  and  cost  growth  and  between  technology  extensions 
and  schedule  growth  in  both  development  and  production  phases 
of  weapons  system  acquisition.  The  objective  here  is  to 
hypothesize  why  such  associations  may  exist. 
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The  hypotheses,  then,  are  based  on  how  three  measures  of 
technology  introduced  in  Chapter  I,  REACH,  ADVANCE,  and  STAND, 
relate  to  cost  growth  and  schedule  slippages.  There  are  five 
distinct  areas  to  be  examined:  three  involving  cost  growth 
and  two  involving  schedule  growth. 

1.  Technological  Complexity  and  Cost  Growth 

The  first  three  hypotheses  rest  on  the  idea  that  there 
is  a  direct  relationship  between  cost  growth  of  a  given  system 
and  the  technological  complexity  of  the  system.  Two  variables 
describe  the  technology  associated  with  a  given  system:  STAND 
and  ADVANCE.  STAND  measures  the  current  state  of  technology 
at  the  time  of  system  development.  This  time  is  defined  as 
the  year  of  Full-Scale  Development  for  each  system.  Thus, 
STAND  measures  the  general  state-of-the-art  of  missile 
technology  at  the  point  when  each  individual  system  is 
developed.  It  is  hypothesized  that  costs  become  more 
difficult  to  control  as  the  degree  of  technological  complexity 
increases.  Hence,  cost  growth  is  expected  to  be  positively 
associated  with  STAND. 

The  second  variable  is  ADVANCE,  which  measures  the 
incremental  change  in  technology  achieved  by  the  development 
program.  Thus,  ADVANCE  measures  the  incremental  jump  in 
technology,  beyond  the  prevailing  state-of-the-art,  associated 
with  the  specific  system  being  created.  It  is  hypothesized 
that  costs  become  more  difficult  to  control  as  the  incremental 


advance  increases.  Hence,  cost  growth  is  expected  to  be 
positively  associated  with  ADVANCE.* 

Three  measures  of  cost  growth  are  examined.  To 
understand  the  differences  between  acquisition  program  phases, 
development  and  production  cost  growth,  representing  two 
distinct  phases  of  DOD's  weapons  system  acquisition  strategy, 
are  separated.  Thus,  development  cost  growth  and  production 
cost  growth  are  two  measures.  The  third  is  total  program  cost 
growth  which  is  the  total  growth  in  costs  for  both  the 
development  and  production  phases.  Although  production  costs 
are  a  much  higher  percentage  of  the  total  than  development 
costs,  development  costs  are  still  important  because  most 
technical  difficulties  are  encountered  in  the  development 
phase  [Ref.  ll:p.  III-7].  The  hypotheses  involving 
cost  growth  are: 

Development  Cost  Growth  =  f (STAND, ADVANCE) 

H2:  Production  Cost  Growth  =  f{ STAND, ADVANCE) 

H3:  Total  Progreun  Cost  Growth  =  f (STAND, ADVANCE) 

2 .  Technological  Complexity  and  Schedule  Slippages 

The  fourth  and  fifth  hypotheses  concern  time,  measured 
as  an  increase  (or  decrease)  of  the  schedule  beyond  the 

*Recall  that  REACH  was  the  third  variable.  Since  REACH 
is  the  sum  of  STAND  and  ADVANCE  there  is  no  additional 
information  and  so,  it  is  excluded  from  the  analysis. 
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original  estimate.  Hypotheses  concerning  schedule  slippage 
are  analogous  to  those  for  cost  growth.  It  is  hypothesized 
that  schedules  become  more  difficult  to  meet  as  the  degree  of 
technological  complexity  increases.  Hence,  schedule  growth  is 
expected  to  be  positively  associated  with  STAND.  It  is  also 
hypothesized  that  schedules  become  more  difficult  to  control 
as  the  incremental  technological  advance  represented  by  the 
particular  system  increases.  Hence,  schedule  growth  is 
expected  to  be  positively  associated  with  ADVANCE. 

Two  measures  of  schedule  growth  are  examined.  As  with 
cost  growth,  development  and  production  phases  are  separately 
viewed.  Total  schedule  growth  is  not  relevant  since  the 
various  development  and  production  contracts  of  a  given 
program  usually  require  dissimilar  and  unrelated  scheduling 
commitments.  The  hypotheses  for  schedule  growth  are: 

H*:  Development  Schedule  Growth  =  f {STAND, ADVANCE) 

H5:  Production  Schedule  Growth  =  f{ STAND, ADVANCE) 

E.  SUMMARY 

This  chapter  has  examined  the  sample  to  be  analyzed  and 
the  sources  of  data.  Using  this  information,  the  next  step  is 
to  develop  technology,  cost,  and  schedule  measures  to  be  used 
to  test  the  hypotheses  presented  in  this  chapter.  Chapter  III 
will  discuss  the  process  of  creating  summary  technology 
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measures  and  will  provide  a  review  of  Tyson's  procedures  for 
creating  cost  and  schedule  measures.  Chapter  IV  will  then 
test  the  hypotheses. 
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III.  TECHNOLOGY.  COST.  AND  SCHEDULE  MEASUREMENT 

In  order  to  test  the  hypotheses  presented  in  the  previous 
chapter,  summary  measures  of  technology,  cost,  and  schedule 
are  required.  The  process  of  creating  these  measures  is  the 
next  step.  Technology  measures  will  be  addressed  first, 
followed  by  cost  and  schedule  measures. 

A.  CREATING  TECHNOLOGY  MEASURES 

In  Chapter  II,  data  sources  were  introduced  and  the 
initial  selection  of  technology  describing  variables  was 
discussed.  In  this  section,  the  process  of  creating  measures 
of  technology  is  continued.  The  first  step  in  this  process  is 
variable  reduction.  The  second  step  is  the  creation  of 
summary  measures  of  technology. 

1 .  Variable  Reduction 

Recall  from  Chapter  II  that  data  were  obtained  on  14 
variables  for  the  15  systems  in  the  sample.  Since  this  many 
variables,  given  a  sample  size  of  only  15,  is  statistically 
unmanageable,  further  variable  reduction  was  necessary.  Three 
methods  of  variable  reduction  were  chosen.  The  first  two  used 
statistical  methods.  The  third  method  involved  a  judgmental 
process  based  on  technical  expertise. 

The  first  method  of  variable  reduction  used  a 
univariate  regression.  This  method  rests  on  the  assumption 
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that  technology  should  increase  with  time  and  that  good 
technology  variables  should  show  a  positive  relationship  to 
time.  Thus,  a  time  regression  is  used  to  identify  the  best 
individual  variables.  The  year  of  development,  which  was 
previously  defined  as  the  FSD  start  date,  was  the  dependent 
variable  in  a  series  of  regression  equations.  Each  of  the  14 
technology  variables  was  used  in  a  separate  regression  as  a 
single  independent  variable. 

Some  results  of  this  initial  series  of  regressions 
were  unexpected.  That  is,  coefficients  of  several  of  the 
variables  were  negative,  when  they  were  expected  to  be 
positive.  This  could  signify  that  these  variables  were  not 
the  indicators  of  technology  that  the  technical  experts 
thought  them  to  be.  Since,  upon  consultation,  they  thought 
this  unlikely,  a  review  of  the  data  values  was  undertaken. 
Three  categories  of  the  variables  describing  missile 
performance — RNG,  ALT,  and  MSR — were  found  to  be  affected  by 
mission  requirements.  Recall  from  the  description  of  the 
sample  that  there  are,  at  least,  three  distinct  groups  of 
tactical  air-launched  munitions  in  the  15  system  sample:  (1) 
air-to-air,  (2)  air-to-surface  (tactical  air-launched) ,  and 
(3)  air-to-surface  (helo  launched).  It  was  determined  that 
design  considerations  of  a  given  system  dictated  certain 
performance  parameters.  This  made  direct  comparisons  among 
the  three  different  groups  difficult,  if  not  impossible. 
Restricting  the  sample  to  just  AIM  or  AGM  series  missiles  was 
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not  considered  an  acceptable  alternative  since  that  would  lead 
to  still  more  problems  due  to  an  increasingly  small  sample 
size  (n  <  15) . 

To  make  the  RNG,  ALT,  and  MSR  variables  comparable 
amongst  all  three  groups,  data  transformation  was  necessary  to 
normalize  the  measures  by  group.  Actual  data  values  were 
converted  to  ratios.  To  accomplish  this,  some  additional  data 
collection  was  necessary.  Actual  values  were  obtained  for  the 
earliest,  operational  version  of  each  missile  system,  if  not 
already  in  the  sample.  Thus,  for  the  AIM-7  Sparrow  and  the 
AIM-9  Sidewinder,  the  "A"  or  "B"  series  values,  representing 
the  earliest  date  of  technology,  were  included  to  compute  the 
ratios.  A  value  of  "1"  was  assigned  to  the  lowest  value, 
representing  the  most  primitive  level  of  technology,  in  each 
of  the  three  mission  groups.  With  the  original  data  value  of 
this  system  in  the  denominator  and  each  of  the  remaining 
system  values,  in  turn,  as  the  numerator,  the  ratios  were  then 
computed  for  each  of  the  systems  in  the  sample.  In  short, 
values  for  RNG,  ALT,  and  MSR  were  normalized  by  dividing  a 
group-specific  baseline  value  for  RNG,  ALT,  and  MSR 
respectively.  Once  the  variables  were  re-configured  into 
ratios,  the  individual  univariate  regressions  were  run  again; 
this  time  with  much  better  results. 

The  next  step  was  to  establish  criteria  for  selection 
of  the  most  significant  variables.  Upon  review  of  the  output. 
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the  criteria  chosen  was  an  greater  than  or  equal  to  .  10  and 
a  P-Value  less  than  or  equal  to  .15.  Five  of  the  14  variables 
met  these  criteria.  They  are  listed  in  Table  III-l.  This 
array  of  variables  was  labelled  Version  A. 


TABLE  III-l 

VARIABLE  REDUCTION; 

UNIVARIATE  REGRESSION 
VERSION  A 

RESULTS , 

Variable 

(%) 

D-Value 

ALT  2 

16.3 

.  136 

TGS2 

18.7 

.  107 

ECCil 

34.8 

.021 

TMCC 

18.3 

.  Ill 

FUZE 

19.4 

.  100 

The  second  method  of  variable  reduction  used  was 
stepwise  regression.  Again  the  dependent  variable  was  FSD 
date.  Also,  again,  the  objective  was  to  identify  a  small  set 
of  variables  most  highly  associated  with  time.  Use  of  the 
normalized  data  obtained  via  the  ratio  transformation  was 
continued  in  this  process.  The  advantage  of  stepwise 
regression  is  that  it  provides  an  ability  to  isolate  a  subset 
of  predictor  (independent)  variables  that  yield  an  optimal 
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equation.  The  stepwise  method  selects  from  a  large  set  of 
candidates  those  independent  variables  that  best  predict  the 
dependent  variable  [Refs.  22,  23].  Both  forward  inclusion  and 
backward  elimination  routines  were  used.  The  best  results 
were  obtained  with  forward  inclusion.  Three  variables  were 
selected  in  this  manner — ALT2,  SPD3,  and  ECCM — with  a 
relatively  high  of  64.96%.  Results  of  this  stepwise 
regression  are  contained  in  Table  III-2  and  is  labelled 
Version  B. 


TABLE  III-2 


VARIABLE  REDUCTION:  STEPWISE  REGRESSION, 

VERSION  B 


Stepwise  regression 

of  FSDSD  on 

14  predictors 

,  with 

Step 

1 

2 

3 

Constant 

69.50 

71.74 

69.07 

ECCM 

8.30 

10.00 

5.00 

T-Ratio 

2.44 

3.39 

1.39 

SPD3 

11.20 

20.00 

T-Ratio 

2 . 04 

3.07 

ALT2 

3.20 

T-Ratio 

2.05 

s 

5.75 

5.16 

4.58 

r2 

34.85 

51.58 

64.96 
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since  neither  the  univariate  nor  the  stepwise  methods 
were  completely  satisfactory  for  use  in  the  Year-of-Technology 
analysis  to  follow,  it  was  decided  that  the  variables  selected 
by  both  methods  should  be  used  and  to  add  a  third  list  of 
variables.  This  third  list  was  derived  using  a  judgmental 
approach  based  on  the  opinion  of  technical  experts  as  to  those 
variables  that  should  best  describe  the  level  of  technology  in 
these  systems.  Several  combinations  of  variables  developed  by 
the  experts  were  tested  by  both  univariate  and  stepwise 
regression  to  ensure  that  they  were  statistically  sound. 
Combinations  that  contained  the  "RNG"  and  •'MSR"  variables 
achieved  unexpected  results  in  the  form  of  negative 
coefficients,  indicating  a  non-positive  relationship  to  the 
passage  of  time,  so  were  dropped  from  further  consideration. 
A  final  version,  acceptable  to  the  experts  and  statistically 
sound,  was  eventually  settled  on.  The  variables  selected  in 
this  manner  are  listed  in  Table  III-3.  This  third  version  was 


labelled  Version  C. 

2 .  Year  of  Technology 

With  three  reduced  sets  of  technology  variables  now 
selected,  the  next  step  was  to  create  summary  measures  that 
describe  the  extension  in  technology  of  each  missile  system. 
The  process  used  was  the  Year  of  Technology  approach.  Recall 
that  the  Year  of  Technology  approach  relates  technology 
measures  to  time  to  predict  the  expected  year  that  the  system 
should  be  produced.  The  difference  between  the  actual  year. 


TABLE  III-3 

VARIABLE  REDUCTION: 
JUDGMENTAL  METHOD, 
VERSION  C 

ALT2  ECCM 

SPD3  IMCC 

TGS2  FUZE 


which  in  our  case  is  the  FSD  start  date,  and  the  expected  or 
computed  year  is  used  as  the  measure  of  the  advance  in 
technology  for  the  individual  system. 

Following  this  process,  the  years  of  FSD  start  of  each 
missile  program  were  separately  regressed  against  the 
variables  selected  in  Versions  A,  B,  and  C.  Results  are 
contained  in  Appendix  C.  Models  B  and  C  explain  a  moderately 
high  portion  of  the  variance  (adjusted  R^  =  55.4%  and  51.9%, 
respectively) .  Version  A  explains  significantly  less 
(adjusted  R^  =  24.2%)  due  in  part  to  the  number  of  predictor 
variables  in  the  model.  Each  coefficient  is  positive  which  is 
consistent  with  the  values  of  the  technology  describing 
variables  reflecting  increasing  technology  through  time.  This 
furnishes  some  confirmation  that  the  variables  selected  in  the 
previous  step  appropriately  reflect  the  extent  of  technology 
and  technology  growth. 
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Tables  III-4,  III-5,  and  III-6  are  plots  of  the  Year 
of  Full-Scale  Development  (Y)  versus  Year  of  Technology  (Y^)  . 
Values  for  STAND,  ADVANCE,  and  REACH  reflecting  technological 
complexity  or  extension  were  then  computed  for  each  system  in 
each  of  the  three  versions.  Results  are  contained  in  Table 

B.  COST  AND  SCHEDULE  MEASURES 

This  section  is  primarily  a  synopsis  of  Tyson's  creation 
of  measures  for  program  cost  and  schedule  growth  outcomes 
[Ref.  11].  As  discussed  earlier,  Tyson  separated  cost  and 
schedule  growth  into  development  and  production  phases. 
Separate,  though  similar,  procedures  were  used  for  cost  and 
schedule  measurement. 

1 .  Cost  Growth  Measures 

The  study  created  cost  growth  ratios  using  the 

following  procedures  and  controls; 

Program  cost  estimates  were  collected  for  each  system. 
A  base-year,  constant  dollar  figure  was  used  so  that 
inflation  would  not  distort  comparisons  among  programs 
that  were  established  at  different  times. 


^There  are  other  methods  for  determining  measures  of 
STAND,  ADVANCE,  and  REACH.  An  alternative  that  Dr.  Moses 
tested  involved  designating  an  "individual  system  as  a 
reference  point.  Candidates  might  be  a)  an  immediate 
predecessor  system  or  b)  the  predecessor  system  with  the 
greatest  REACH  (maximum  predecessor  technology) .  The 
technology  embodied  in  either  reference  system  would 
constitute  STAND,  and  ADVANCE  would  be  measured  as  deviations 
from  the  specific  reference  system. . .These  alternatives  were 
explored  with  no  material  enhancement  of  the  analysis."  [Ref. 
2:p.  17fn] 
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TABLE  III-4 

YEAR  OF  TECHNOLOGY  PLOT 
VERSION  A 


55  60  65  70  75  80  85 


Year  of  Full-Scale  Development 
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TABLE  III-5 

YEAR  OF  TECHNOLOGY  PLOT 
VERSION  B 


Year  of  Technology 


55  60  65  70  75  80  85 


Year  of  Full-Scale  Development 
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TABLE  III-6 

YEAR  OF  TECHNOLOGY  PLOT 
VERSION  C 


TABLE  III-7 


TECHNOLOGY  MEASURES 


SYSTEM  STAND 

A-ADV  A 

-REACH 

B-ADV  B- 

-REACH 

C-ADV  C- 

-REACH 

AIM-7E 

60 

5.83 

65.83 

1.39 

61.39 

-0.74 

59.26 

AIM-7F 

66 

-0.55 

65.45 

0.23 

66.23 

-1.26 

64.74 

AIM-7M 

78 

-4.14 

73.86 

3.70 

81.70 

0.57 

78.57 

AIM-9L 

71 

1.15 

72.15 

0.74 

71.74 

1.46 

72.46 

AIM-9M 

76 

1.04 

77.04 

0.70 

76.70 

-1.78 

74.22 

AIM-54A 

62 

5.86 

67.84 

0.23 

62.23 

1.83 

63.83 

AIM- 54 C 

77 

4.09 

81.09 

0.30 

77.03 

2.86 

79.86 

AGM-88A 

78 

-6.98 

71.02 

-5.76 

72.24 

-3.29 

74.71 

AGM-84A 

73 

-1.19 

71.81 

-0.76 

72.24 

-0.71 

72.29 

AGM-65A 

/B 

68 

-2.55 

65.45 

4.24 

72.24 

2.83 

70.83 

AGM-65D 

/G 

76 

3.09 

79.09 

1.20 

77.20 

1.21 

77.21 

AIM-120A 

82 

-4.09 

77.91 

-5.30 

76.70 

-2.86 

79.14 

AGM-114A 

76 

-4.19 

71.81 

-3.76 

72.24 

-3.71 

72.29 

BGM-7 lA 
/B 

63 

8.81 

71.81 

9.24 

72.24 

9.29 

72.29 

BGM-7 ID 

78 

-6.19 

71.81 

-5.76 

72.24 

-5.71 

72.29 
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Development  costs  were  obtained  for  the  period  from 
program  start  to  initial  operating  capability  (IOC)  date. 
Those  development  costs  after  IOC  were  excluded  because 
these  costs  generally  consisted  of  modifications  or  other 
changes  not  within  the  scope  of  the  original  development 
project. 

Development  Cost  Growth  (DCG)  ratios  were  then  calculated  by 
dividing  the  actual  development  cost  at  IOC  by  the  development 
cost  estimate  at  FSD  start  using  the  following  formula: 

^  _ Development  Cost  at  IOC _ 

Development  Cost  Estimate  at  FSD 

For  production  cost  growth,  there  were  additional 
considerations.  The  first  was  that  the  data  for  production 
cost  represented  the  cost  to  the  government,  not  simply  the 
overall  cost  of  the  project.  Therefore,  production  cost 
growth  ratios  more  accurately  refer  to  price  growth.  Second, 
the  study  took  account  of  changes  in  quantities  procured  as 
programs  progressed.  Adjustments  to  costs  included  charges 
for  schedule,  engineering,  inflation,  and  estimating  changes. 
Using  a  "price- improvement  methodology"  that  included  these 
controls,  Tyson  created  price-improvement  learning  curves  to 
calculate  the  cost  of  the  originally  planned  quantity.  The 
production  cost  growth  (PCG)  ratio  was  based  on  a  comparison 
of  the  original  estimate  and  the  later,  revised  cost  at  the 
original  planned  quantity.  The  production  cost  growth  formula 
can  be  stated  as: 

_  _ Production  Cost  at  Production  Start _ 

Adjusted  Estimate  of  Production  Cost  (Original  Qty) 
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The  final  cost  outcome  measure,  Total  Program  Cost 
Growth  (TPCG) ,  was  then  determined.  This  ratio  was  the  sum  of 
the  estimate  of  development  costs  at  IOC  and  production  cost 
at  production  start  divided  by  the  sum  of  the  development  cost 
estimate  at  FSD  plus  the  adjusted  estimate  of  production  costs 
based  on  the  original  quantity.  The  formula  was: 

TPCG  =  Cost  at  IOC  +  Pr .  Cost  at  Start 

Dev.  Cost  at  FSD  +  Adjusted  Pr .  Cost 

2 .  Schedule  Measures 

Providing  measures  of  schedule  slippage  was  less 
exacting.  The  formula  that  Tyson  used  for  development 
schedule  growth  is  simply  a  measure  of  time.  The  ratio  was 
computed  by  dividing  the  actual  span  in  months  from 
development  start  (FSD)  to  IOC  by  the  original  estimate  prior 
to  FSD.  The  formula  used  to  compute  the  development  schedule 
growth  (DSG)  ratio  was; 

DSG  =  Actual  FSD  -  IOC  {Months) 

Estimated  FSD  IOC  (Months) 

Production  schedule  growth  was  measured  using  the  same 
technicpae.  The  production  span  was  defined  as  the  period  from 
contract  award  to  the  end  of  production  or,  if  still  in 
production,  the  last  year  of  planned  funding.  The  production 
schedule  growth  (PSG)  ratio  formula  is: 


PSG 


Actual  Production  Start  -  End  (Months) 
Estimated  Production  Start  -  End  (Months) 
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Actual  values  of  the  ratios  from  the  Tyson  study  for  these 


five  program  outcomes  are  contained  in  Table  III-8. 


TABLE 

III-8 

OUTCOME 

MEASURES 

SYSTEM 

DCG 

DSG 

TPCG 

PCG 

PSG 

AIM-7E 

0.84 

1.00 

1.07 

1.08 

3.11 

AIM-7F 

4.25 

3.90 

1.74 

1.58 

1.93 

AIM-7M 

0.98 

1.46 

1.29 

1.31 

1.61 

AIM-9L 

4.89 

2.45 

2.25 

2.07 

2.76 

AIM-9M 

2 . 04 

1.01 

1.10 

1.01 

2.44 

AIM-54A 

1.54 

1.07 

1.39 

1.35 

1.20 

AIM-54C 

1.67 

1.45 

1.93 

2.01 

3.71 

AGM-88A 

2.03 

1.05 

1.47 

1.39 

1.61 

AGM-84A 

1.17 

1.36 

1.53 

1.64 

3.05 

AGM-65A/B 

1.15 

1.46 

0.95 

0.84 

2.17* 

AGM-65D/G 

1.07 

1.98 

1.53 

1.58 

2 . 14 

AIM-120A 

1.44 

1.80 

1.06 

1.05 

1.11 

AGM-114A 

1.09 

1.44 

1.39 

1.61 

2 . 38 

BGM-71A/B 

1.20 

1.45 

1.70 

1.78 

2.27 

BGM-71D 

1.70 

1.02 

0.98 

0.95 

0.94 

*  PSG  for  the  AGM-65A/B  Maverick  was  not  provided  in  the 
Tyson  study's  acquisition  program  database.  The  mean  value 
of  PSG  was  inserted  to  preclude  a  distortion  of  the  data 
for  use  later  in  the  analysis. 

Source:  [Ref.  11 

: App .  A ] 
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C.  SUMMARY 

This  chapter  has  reviewed  Tyson's  process  of  creating 
measures  of  cost  growth  and  schedule  slippage.  Using 
techniques  from  prior  studies,  measures  of  technological 
complexity  and  extension  have  also  been  created.  With  these 
key  building  blocks  in  place,  the  next  step  is  to  begin  the 
process  of  examining  relationships  between  technology  and  cost 
and  schedule  growth. 


IV.  STATISTICAL  ANALYSIS  AND  TESTS 


The  purpose  of  this  chapter  is  to  describe  the  analysis  of 
the  hypotheses  regarding  cost  and  schedule  growth  and  their 
relationship  to  technological  complexity  and  extension.  In 
the  previous  chapter,  techniques  developed  in  previous  studies 
were  used  to  create  measures  of  technology.  Procedures  and 
data  used  in  the  Tyson  study  to  create  measures  of  cost  and 
schedule  growth  were  also  examined.  Having  developed  these 
measures,  the  main  purpose  of  this  thesis  is  to  test  for  the 
existence  of  relationships  between  cost  and  schedule  growth 
and  the  level  of  technology  embodied  in  systems.  This  is  to 
be  accomplished  by  statistically  testing,  using  regression 
analysis,  the  five  hypotheses  presented  in  Chapter  II. 

This  chapter  is  organized  into  four  sections.  The  first 
section  will  discuss  methods  to  examine,  evaluate,  and  prepare 
the  variables  to  be  used  in  the  regression  analysis.  The 
second  section  will  discuss  the  procedures  used  in  the 
regression  analysis  itself  and  will  review  results.  Based  on 
these  results,  the  third  section  will  explore  some  approaches 
to  improve  upon  those  results,  including  the  introduction  of 
control  factors.  The  final  section  will  summarize  the 
results. 
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A.  ANALYSIS  OF  VARIABLES 


Prior  to  testing  the  hypotheses,  the  variables  required 
examination  to  determine  their  suitability  for  use  in  the 
regression  analysis.  The  first  step  was  to  check  for 
normality  and  multicollinearity,  identify  possible  outliers  in 
the  data,  and  highlight  any  variables  that  might  need 
transformation  [Refs.  23,24].  Several  outliers  were 

discovered. 

1 .  Outliers . 

The  following  alternatives  for  processing  outliers 
were  considered. 

a.  Disregard  Outliers 

Simply  put,  this  means  to  include  the  outliers  in 
the  regression  analysis.  Since  outliers  can  have  a 
significant  impact  on  correlat;on  and  regression  tests,  this 
was  rejected. 

b.  Discard  Outliers 

This  procedure  eliminates  the  detrimental  impact 
on  the  tests.  This  too  was  rejected,  however,  since  to  delete 
these  observations  would  result  in  substantially  reducing  the 
sample  size.  This  was  unacceptable  considering  the  already 
small  number  of  systems  in  the  sample. 

c.  Data  Transformation 

There  are  several  methods  for  transforming  data 
values  for  improved  fit.  These  include  square-root,  inverse 
function,  power,  log,  exponential,  and  ratio  transformation. 
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However,  since  the  data  consisted  of  a  mixture  of  variables 
(values  were,  in  some  cases,  negative,  some  positive,  some 
fractional  greater  than  one,  and  some  fractional  less  than 
one) ,  no  one  method  was  mathematically  feasible.  Therefore, 
this  method  was  also  rejected. 

d.  Truncate  Outliers 

Truncating  outliers  involves  establishing  maximum 
and  minimum  data  values,  or  boundaries,  for  each  of  the 
variables.  Outliers  are  first  identified  by  observation  of 
the  distribution.  Maximum  and  minimum  values  are  set  using 
somewhat  subjective  methods  to  define  a  "reasonable  range"  for 
the  distribution.  Values  falling  above  or  below  this  range 
are  then  truncated  to  the  minimum/maximum  boundaries  of  the 
range.  In  other  words,  outliers  values  are  pulled  in  to  form 
a  more  compact  distribution.  Since  outliers  were  identified 
for  a  number  of  variables,  this  method  was  chosen. 

2 .  Processing  Outliers 

The  "reasonable  range"  for  outliers  was  identified  by 
reviewing  box  and  stem-and-leaf  plots  of  the  variables.  Data 
values  more  than  one  and  one-half  times  the  normal 
distribution  box-plot  length  (+  or  -)  from  the  extreme  values 
of  the  middle  50%  of  the  data  (essentially  the  data  values 
residing  at  or  nearest  to  the  first  and  third  quartiles)  were 
selected  for  truncation.  This  represented  values  well  outside 
the  normal  distribution  range  (approximately  150%  greater  than 
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or  less  than  the  nearest  non-outlier  value). The  following 
approach  was  used  to  assign  new  values  to  truncated  data 
values . 

a.  High  Outliers 

DCG  for  the  AIM-7F  (refer  to  Table  III-8) ,  by  way 
of  example,  was  identified  as  one  of  a  number  of  high 
outliers.  The  highest  and  second  highest  non-outliers  were 
then  identified.  The  outlier  value  was  truncated  at  twice  the 
percentage  diffi^rence  between  the  two  highest  non-outliers. 
The  following  formula  was  used  to  assign  truncated  values 
[Ref.  24:p.  61]: 

Ojj  =  [  (  (Djj  —  DH.i/Dg.i)  X2)  +1]  xDg 

where : 

Or  =  The  outlier  replacement  value 

Dr  =  The  highest  non-outlier  value 

Dr-i  =  The  second  highest  non-outlier  value 

The  computed  value  for  Or  then  replaces  the  original  value  for 
use  in  the  subsequent  regression  analysis.^  Since  this 
procedure  resulted  in  some  truncated  values  still  falling 
outside  our  "reasonable  range,"  two  additional  controls  were 
employed.  A  subjective  limit  of  20%  was  set  to  limit  the 

^This  method  represents  the  default  procedure  of  outlier 
selection  in  MINITAB  [Ref.  22:p.  210]. 

^Using  the  DCG  example.  Dr  =  2.04  and  Dr.j  =  2.03,  then: 
Or  [((  =  [((2.04  -  2.03/2.03)  X  2)  +1]  X  2.04  =  2.06.  The 
truncated  data  value  is  then  established  as  2.06. 
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extremes  of  truncated  values.  Also,  if  more  than  a  10^ 
difference  existed  between  the  two  highest  non-outliers,  then 
the  outlier  value  was  replaced  with  the  lower  oi  Or  or  the 
value  20%  larger  than  the  highest  non-outlier. 

b.  Low  Outliers 

Although  there  were  no  low  outliers  were 
identified,  they  would  be  processed  using  a  similar  technique. 
The  following  formula  was  used  [Ref.  24:p.  62]: 

Or  =  [1-((Dl-i  -  Dl/Dl-i)  X  2)]  +1 

where: 

Or  =  Same  as  defined  above 

Dl  =  The  lowest  non-outlier 

Dl.i  =  The  second  lowest  non-outlier. 

As  above,  the  value  of  Or  replaced  the  original  outlier  values. 
Similar  controls  were  also  used. 

c .  Summary 

The  outliers  that  were  "pulled  in"  using  these 
procedures  were  the  B-ADVANCE  and  C-ADVANCE  values  of  the  BGM- 
71A/B  TOW  missile,  DCG  and  DSG  of  the  AIM-7F  Sparrow  missile, 
and  DCG  of  the  AIM-9L  Sidewinder  missile.  With  the  data 
values  for  both  dependent  and  independent  variables  now 
"cleaned,"  the  actual  analysis  was  begun. 
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B.  TESTS  OF  HYPOTHESES 

ThP  next  step  was  to  test  the  hypotheses.  This  was  done 
using  multiple  regression  with  Tyson's  measures  of  cost  and 
schedule  growth  as  single  dependent  variables.  The  measures 
of  technological  complexity  and  extension  (STAND  and  ADVANCE) , 
in  three  separate  versions,  were  the  independent  variables. 
The  dependent  and  independent  variables  are  summarized  in 
Table  IV-1.  As  explained  in  Chapter  II,  REACH  was  not  used  in 
this  initial  series  of  regressions.  Results  of  the 
regressions  are  contained  in  Tables  IV-2  through  IV-6. 


TABLE  IV-1 

TECHNOLOGY 

AND  OUTCOME  MEASURES 

TECHNOLOGY 

OUTCOMES 

flndeoendent  Variables) 

fDeoendent  Variables) 

STAND  (Current  State) 

Development  Cost  Growth 

ADVANCE  (Extension) 

Production  Cost  Growth 

REACH  (Amount) 

Total  Program  Cost  Growth 

Development  Schedule  Growth 

Production  Schedule  Growth 
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TABLE  IV- 2 


DEVELOPMENT  COST  GROWTH, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 

Variable  Variable _ COEFF  t _ p-Value  Statistics 


DCG 

Constant 

STAND 

A-ADV 

0.9830 

0.0067 

-0.0105 

0.54 

0.27 

-0.29 

0.597  F 

0.792  R2  = 

0.775  Adj .r2  = 
P 

.26 

4.2% 

0.0% 

.774 

DCG 

Constant 

1.5610 

1.04 

0.317 

F 

.88 

STAND 

-0.0014 

-0.07 

0.947 

r2  = 

12.8% 

B-ADV 

-0.0^61 

-1.13 

0.279  Adj.R^  = 

0.0% 

P 

.439 

Constant 

0.9990 

0.69 

0.506 

F 

.  38 

STAND 

0.0064 

0.31 

0.759 

R2  = 

6.0% 

C-ADV 

-0.0285 

-0.56 

0.586 

Adj  .R^  = 

P 

0.0% 
.  692 
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TABLE  IV- 3 


PRODUCTION  COST  GROWTH, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 

Variable  Variable  COEFF  t  p-Value  Statistics 


PCG 


Constant 

STAND 

A-ADV 


0.137  -0.09 
0.021  1.08 
0.051  1.79 


0.926  F 

0.300  R2 

0.099  Adj .R^ 
P 


1.62 

21.3% 

8.2% 

.238 


Constant 

1.229 

0.89 

0.392 

F 

.  17 

STAND 

0.003 

0.14 

0.892 

R2  = 

2.8% 

B-ADV 

0.021 

0.55 

0.593 

Adj  .  R^  = 

0.0% 

P 

.843 

Constant 

0.714 

0.60 

0.558 

F  =  1.44 

STAND 

0.010 

0.61 

0.550 

R2  =  19.4% 

C-ADV 

0.070 

1.68 

0.118 

Adj  .r2  =  6.0% 

P  =  .274 
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TABLE  IV-4 

TOTAL  PROGRAM  COST  GROWTH 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 


Variable 

Variable 

COEFF 

t 

D-Value  Statistics 

TPCG 

Constant 

STAND 

A-ADV 

0.405 

0.014 

0.041 

0.28 

0.70 

1.43 

0.786 

0.496 

0.178 

F 

R^  = 
Ad  j  .  r2  = 
P 

1.11 

15.6% 

1.5% 

.362 

TPCG 

Constant 

1.4530 

1.08 

0.303 

F 

.22 

STAND 

-0.0003 

-0.02 

0.987 

R2  = 

3.5% 

B-ADV 

0.0196 

0.54 

0.602 

Adj.R^  = 

0.0% 

P 

.807 

TPCG  Constant  0.940  0.82  0.431  F  =  1.51 

STAND  0.007  0.44  0.666  R^  =  20.1% 

C-ADV  0.0679  1.69  0.118  Adj.R^  =  6.8% 

P  =  .260 
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TABLE  IV- 5 


DEVELOPMENT  SCHEDULE  COST  GROWTH, 
MULTIVARIATE  REGRESSIONS 


Dependent 

Variable 

Independent 

Variable 

COEFF 

t 

D-Value 

Model 

Statistics 

DSG 

Constant 

1.123 

0.55 

0.595 

F  =  0.02 

STAND 

0.005 

0.18 

0.858 

R2  =  0.40% 

A-ADV 

0.008 

0.04 

0.840  Adj 

.r2  =  0.00% 

P  =  .978 

Constant 

0.697 

0.40 

0.697 

F 

0.27 

STAND 

0.111 

0.46 

0.651 

R2  = 

4.4% 

B-ADV 

0,035 

0.74 

0.474 

Adj.R^  = 

0.0% 

P 

.764 

DSG  Constant  0.6450  0.40  0.696  F  =  0.52 

STAND  0.0121  0.54  0.598  R^  =  8.00% 

C-ADV  0.0573  1.02  0.328  Adj.R^  =  0.00% 

P  =  .606 
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TABLE  IV- 6 


PRODUCTION  SCHEDULE  COST  GROWTH, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent 
Variable  Variable _ COEFF 


Model 

>-Value  Statistics 


PSG  Constant  0.337  0.10  0.921  F  =  1.85 

STAND  0.034  0.75  0.470  R^  =  25.1% 

A-ADV  0.113  1.74  0.109  Adj.R^  =  11.5% 

P  =  .203 


PSG 

Constant 

STAND 

B-ADV 

2.049 

0.002 

0.103 

0.74 

0.06 

1.29 

0.475 

0.953 

0.223 

F 

R2  = 
Adj.R^  = 
P 

1.13 

17.1% 

2.0% 

.357 

PSG 

Constant 

2 . 19700 

0.88 

0.396 

F 

1.25 

STAND 

0.00003 

0.00 

0.999 

R^  = 

17.2% 

C-ADV 

0.12067 

1.38 

0.191 

Ad  j  .  r2  = 
P 

3.5% 

.321 

L .  Development  Cost  Growth  and  Technological  ComDlexit\ 
Recall  that  the  first  hypothesis  is  that  there  is  a 
direct  relationship  between  cost  growth  in  the  development 
phase  (DCG)  and  the  scope  of  the  development  task  measured  in 
terms  of  technological  complexity.  The  regression  results  are 
mediocre.  Coefficients  for  STAND  and/or  ADVANCE  are  negative 
in  all  three  versions  which  is  unexpected.  The  low  values  for 
and  the  poor  significance  levels  render  this  model 
essentially  without  much  meaning. 

2 .  Production  Cost  Growth  and  Technological  Complexity 
The  second  hypothesis  is  that  production  cost  growth 

(PCG)  is  directly  related  to  technological  complexity.  The 
results  of  these  multivariate  regressions  are  somewhat  better 
than  for  DCG.  The  coefficients  are  positive  as  expected  and 
the  R^  values  for  Versions  A  and  C  are  much  higher.  As  a 
whole,  however,  the  models  are  not  significant  at  traditional 
levels . 

3 .  Total  Program  Cost  Growth  and  Technological 
Complexity 

With  disappointing  results  for  DCG  and  PCG,  it  was  not 
surprising  that  the  results  obtained  for  total  program  cost 
growth  (TPCG)  were  also  unimpressive.  Except  for  Version  B, 
coefficients  were  positive  as  expected.  R^  values  were  lower 
than  for  PCG,  but  closer  to  PCG  than  to  DCG.  This  may  reflect 
the  greater  influence  of  production  costs  than  development 
costs  on  total  program  costs,  as  put  forward  by  Tyson. 
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Somewhat  surprisingly,  the  and  adjusted  R^  values  for 
Version  C  increased. 

4 .  Development  Schedule  Growth  and  Technological 

Complexity 

The  fourth  hypothesis  suggested  a  direct  relationship 
between  development  schedule  growth  (DSG)  and  the  STAND  and 
ADVANCE  measures  of  technology.  Coefficients  were  positive  in 
all  three  versions,  but  again,  the  models  were  insignificant. 

5 .  Production  Schedule  Growth  and  Technological 

Complexity 

The  fifth  and  final  hypothesis  is  that  a  direct 
relationship  exists  between  production  schedule  growth  (PSG) 
and  technology  STAND  and  ADVANCE.  Perhaps  the  best  overall 
results  were  obtained  from  these  regressions.  Coefficients 
were  positive  in  all  three  versions.  The  R^s  were  relatively 
high  compared  to  the  previous  models.  The  highest  adjusted  R^ 
of  all  of  the  models  was  obtained  from  Version  A.  Yet  again, 
however,  the  models  are  not  significant  at  traditional  levels. 

C.  OTHER  CONSIDERATIONS 

1 .  Additional  Tests 

The  relatively  poor  results  obtained  from  the 
regressions  suggested  a  return  to  the  data  transformation 
process  discussed  earlier  in  this  chapter  (see  Section  A.l.c). 
This  was  done  in  an  effort  to  improve  the  outcomes  of  the 
regressions.  Each  of  the  data  transformation  techniques  was 
tried,  in  turn  and  in  combination — where  mathematically 
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feasible — on  both  the  dependent  and  independent  variables. 
Multiple  regressions  were  then  rerun  on  the  transformed 
variables.  Despite  the  multitude  of  regression  run  in  this 
manner,  in  no  case  was  there  a  material  enhancement  of  the 
models. 

Univariate  regressions  were  also  explored.  STAND, 
ADVANCE,  and  REACH  were  separately  used  as  single  independent 
variables.  Regressions  were  run  on  the  original  data  as  well 
as  transformed  data  in  all  combinations.  Again,  no  material 
improvement  was  noted. 

2 .  Control  Factors  That  Affect  Cost  Growth 

Since  the  results  to  this  point  were  not  particularly 
satisfying,  a  new  direction  was  explored.  The  Tyson  study 
identified  several  factors  that  affect  cost  growth.  The  most 
significant  were  program  stretch  and  several  acquisition/ 
contracting  initiatives  such  as  multi-year  procurement, 
competition,  prototyping,  design-to-cost,  total  package 
procurement,  fixed  price  development,  and  contract  incentives. 
Tyson's  factors  are  summarized  in  Appendix  D. 

In  discussing  program  stretch,  the  Tyson  study 
reasoned  that  DOD  and  Congress  "have  sometimes  met  budgetary 
constraints  by  stretching  out  the  production  schedule,  buying 
the  same  quantity  over  a  longer  scheaule,  or  buying  a  lessor 
quantity  over  the  same  period."  [Ref.  ll;p.  V-1]  They 
hypothesized  that  program  stretch,  measured  as  schedule  growth 
divided  by  quantity  growth,  contributes  to  cost  growth.  They 
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performed  a  series  of  univariate  regressions  with  cost  growth 
— development,  production,  or  total  program — as  the  dependent 
variable  and  stretch  as  the  independent  variable.  Their 
findings  indicated  that  program  stretch  was  a  strong,  positive 
determinant  of  both  production  and  total  program  cost  growth, 
but  not  development  cost  growth. 

Tyson  then  examined  whether  acquisition/contracting 
initiatives  were  associated  with  lower  cost  growth.  They 
again  used  univariate  regression  analysis  with  cost  growth  as 
the  dependent  variable  and  the  acquisition/contract  initiative 
as  the  independent  variable.  Their  findings  indicated  the 
following  relationships: 

Fixed  price  development  contracts  (FPD)  were  positively 
related  and  contract  incentives  for  FSD  (I-FSD)  were 
negatively  related  to  development  cost  growth  (DCG) . 

-  Total  package  procurement  (TPP)  was  positively  related 
and  contract  incentives  in  the  production  phase  (I-PRD) 
were  negatively  related  to  production  cost  growth  (PCG) . 

Total  package  procurement  (TPP)  was  positively  related 
while  both  FSD  (I-FSD)  and  production  (I-PRD)  contract 
incentives  were  negatively  related  to  total  program  cost 
growth  (TPCG) . 

Tyson's  finding  suggested  new  tests.  When  program 
stretch  and  acquisition/contracting  factors  are  controlled  for 
the  technology  measures,  STAND  and  ADVANCE  could  prove  useful 
as  predictors  of  cost  growth.  The  following  new  models,  using 
Tyson's  variables  as  controls,  were  developed^: 


^Actual  data  values  for  the  control  factors  are  contained 
in  Appendix  E. 
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Hg:  DCG  =  f{ STAND, ADVANCE, I-FSD,FPD} 

H7:  PCG  =  f {STAND, ADVANCE, I-PRD,TPP, STRETCH) 

He:  TPCG  =  f {STAND, ADVANCE, I-FSD, I-PRD,TPP, STRETCH) 

where : 

I-FSD,  FPD,  I-PRD,  TPP  and  STRETCH  are  as  defined  above. 

Multivariate  regressions  were  performed  to  test  the 
new  models.  Results  are  contained  in  Tables  IV-7,  IV-8,  and 
IV-9.  The  DCG  models  were,  once  again,  unimpressive.  The 
best  results  were  obtained  for  production  cost  growth  and 
total  program  cost  growth  using  Version  C  measures  of  STAND 
and  ADVANCE.  While  the  overall  models  are  not  significant, 
the  t-statistics  and  p-values  for  ADVANCE  are  meaningful. 
This  tends  to  indicate  that,  when  STRETCH  and  acquisition/ 
contracting  factors  are  controlled,  ADVANCE  is  useful  as  a 
predictor  of  production  and  total  program  cost  growth. 

D.  RESULTS 

Low  significance  levels  and  the  small  explanation  of 
variability  (R^)  for  the  initial  models  was  a  common  result. 
The  low  explanatory  ability  of  all  the  DCG  models  was  not 
entirely  unexpected.  Both  Dodson  [Ref.  7]  and  Moses  [Ref.  3] 
used  technology  variables  in  an  attempt  to  predict  development 
cost  (using  direct  dollar  measures  of  development  cost)  with 
little  success.  While  they  attributed  their  poor  results  to 
incomplete  modelling,  the  similarly  poor  results  here  with 
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TABLE  IV- 7 

DEVELOPMENT  COST  GROWTH  WITH  CONTROLS, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 

Variable  Variable _ COEFF  t _ p-Value  Statistics 


DCG 


Constant 

0.671 

0.32 

0.753 

F 

0.19 

STAND 

0.013 

0.03 

0.675 

R^  = 

6.9% 

A-ADV 

-0.013 

0.04 

0.752 

Adj.R2  = 

0.0% 

I-FSD 

-0.146 

-0.44 

0.668 

P 

.941 

FPD 

-0.324 

-0.50 

0.630 

DCG 


Constant 

1.184 

0.75 

0.473 

F 

0.72 

STAND 

0.007 

0.31 

0.764 

R2  = 

22.3% 

B-ADV 

-0.067 

-1.45 

0.177 

Ad  j  .  r2  = 

0.0% 

I-FSD 

-0.287 

-0.90 

0.387 

P 

.598 

FPD 

-0.673 

-1.04 

0.321 

DCG 


Constant 

0.645 

0.39 

0.706 

F  =  0.29 

STAND 

0.013 

0.56 

0.589 

r2  =  10.3% 

C-ADV 

-0.040 

-0.70 

0.499 

Adj  .R^  =  0.0% 

I-FSD 

-0.197 

0.59 

0.568 

P  =  .879 

FPD 

-0.418 

-0.64 

0.538 
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TABLE  IV- 8 


PRODUCTION  COST  GROWTH  WITH  CONTROLS, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 


Variable 

Variable 

COEFF 

t 

D-Value 

Statistics 

PCG 

Constant 

-0.546 

-0.27 

0.793 

F  =  1.15 

STAND 

0.023 

0.92 

0.382 

r2  =  39.0% 

A-ADV 

0.039 

1.26 

0.239  Adj 

.R2  =  5.2% 

I -PRD 

0.310 

1.16 

0.276 

P  =  .401 

TPP 

-0.172 

0.49 

0.732 

STRETCH 

0.030 

0.23 

0.826 

PCG 


Constant 

0.469 

0.27 

0.796 

F 

0.97 

STAND 

0.011 

0.49 

0.635 

r2  = 

35.1% 

B-ADV 

0.037 

0.97 

0.356 

Adj.R^  = 

0.0% 

I -PRD 

0.251 

0.90 

0.392 

P 

.482 

TPP 

-0.560 

-1.11 

0.297 

STRETCH 

0.023 

0.17 

0.870 

PCG 


I 


Constant 

0.263 

0.18 

0.858 

F  =  2.24 

STAND 

0.015 

0.86 

0.414 

r2  =  55.4% 

C-ADV 

0.091 

2.34 

0.044 

Adj  .r2  =  30.6% 

I -PRD 

0.124 

0.52 

0.617 

P  =  .139 

TPP 

-0.783 

-1.83 

0.100 

STRETCH 

0.039 

0.35 

0.735 
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TABLE  IV-9 


TOTAL  PROGRAM  COST  GROWTH  WITH  CONTROLS, 
MULTIVARIATE  REGRESSIONS 


Dependent  Independent  Model 

Variable  Variable _ COEFF  t _ p-Value  Statistics 


TPCG 

Constant 

0.180 

0.09 

0.933 

F 

0.90 

STAND 

0.013 

0.51 

0.623 

R2 

= 

40.4% 

A-ADV 

0.044 

1.26 

0.244 

Ad  j  .  R^ 

= 

0.0% 

I-FSD 

0.266 

0.87 

0.411 

P 

= 

.  537 

I -PRD 

0.183 

0.54 

0.606 

TPP 

0.015 

0.03 

0.977 

STRETCH 

-0 . 020 

-0.15 

0.886 

TPCG 

Constant 

1.161 

0.61 

0.558 

F 

0.73 

STAND 

0.001 

0.05 

0.959 

R^ 

= 

35.3% 

B-ADV 

0.037 

0.91 

0.389 

Ad  j  .  r2 

= 

0.0% 

I-FSD 

0.161 

0.54 

0.603 

P 

= 

.642 

I -PRD 

0.194 

0.54 

0.603 

TPP 

-0.402 

-0.77 

0.464 

STRETCH 

-0.023 

-0.16 

0.874 

TPCG 

Constant 

1.253 

0.86 

0.412 

F 

2 . 05 

STAND 

0.002 

0.09 

0.934 

R2 

= 

60.6% 

C-ADV 

0.105 

2.55 

0.034 

Ad  j  .  R^ 

= 

31.1% 

I-FSD 

0.350 

1.42 

0.193 

P 

= 

.  170 

I -PRD 

-0.088 

-0.29 

0.780 

TPP 

-0.679 

-1.62 

0.145 

STRETCH 

-0.012 

-0.11 

0.915 
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development  cost  growth  offer  some  confirmation  that 
technology  is  not  a  direct  predictor  of  cost  outcomes  in  the 
development  phase. 

The  results  obtained  while  using  control  variables  in 
addition  to  the  technology  variables  to  predict  cost  growth 
were  improved,  indicating  that  ADVANCE  is  potentially  a  useful 
measure  for  estimating  production  and  total  program  cost 
growth . 

Chapter  V  will  summarize  the  thesis  and  present 
conclusions  based  on  the  results  obtained  in  this  chapter. 
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V.  SUMMARY  AND  CONCLUSIONS 


A.  SUMMARY 

This  thesis  has  explored  hypothesized  relationships 
between  technological  complexity  and  both  cost  growth  and 
schedule  growth  in  the  development  and  production  phases  of 
weapons  system  acquisition.  A  key  objective  was  to  develop 
measures  of  technology,  cost  growth,  and  schedule  growth  that 
would  be  useful  for  conducting  a  statistical  analysis  of 
possible  relationships. 

The  first  step  was  to  identify  technology  describing 
properties,  characteristics,  and  performance  measures  for  the 
sample  of  tactical  air-launched  munitions.  With  the 
assistance  of  technical  experts,  data  was  obtained  for  14 
variables  measuring  range,  speed,  altitude,  electronics,  and 
reliability.  To  make  the  number  of  variables  more  manageable 
for  the  subsequent  analysis,  the  set  of  variables  was  reduced 
by  three  separate  methods:  univariate  regression,  stepwise 
regression,  and  a  judgmental  selection  method,  labelled 
Versions  A,  B,  and  C.  All  subsequent  analysis  was  conducted 
three  times,  once  based  on  each  of  the  three  reduced  sets  of 
variables . 

Using  the  Year  of  Technology  Approach  established  in 
previous  studies,  the  reduced  set  of  technology  describing 
variables  was  employed  to  determine  the  level  of  technology 
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embodied  in  each  missile  system.  The  set  of  technology 

measures  were  regressed  against  the  actual  year  in  which  the 

systems  were  developed.  The  resulting  regression  model 

produced  three  measures  of  technology  for  each  missile  system: 

STAND — The  actual  year  in  which  the  missile  system  was 
developed  which  measures  the  current  state  of  technology. 

-  ADVANCE — The  extension  of  technology  beyond  the  current 
state  of  the  art  for  a  given  system. 

REACH — The  total  level  of  technology  embodied  in  each 
system. 

Measures  of  program  outcomes  developed  by  Tyson,  Nelson, 
Om,  and  Palmer  [Ref.  11]  were  used  as  cost  growth  and  schedule 
growth  measures.  In  their  study,  Tyson  et  al.  created  ratio 
measures  of  cost  growth  by  dividing  the  actual  development  or 
production  costs  by  the  costs  estimated  at  the  beginning  of 
the  project.  Similarly,  schedule  growth  ratios  were 
calculated  by  dividing  the  actual  length  of  the  development  or 
production  project  by  the  time  estimated  at  the  beginning  of 
the  project.  In  both  cases,  certain  controls  were  introduced 
to  ensure  accuracy  in  the  ensuing  ratios. 

The  next  step  was  to  statistically  test  for  the  existence 
of  relationships  between  cost  and  schedule  growth  and 
technological  complexity.  Each  of  the  measures  of  cost  growth 
and  schedule  growth  were  regressed  on  STAND  and  ADVANCE.  The 
results  of  these  initial  regressions  were  unimpressive. 

In  an  effort  to  improve  upon  these  models,  data  transfor¬ 
mation  techniques  were  employed.  Various  combinations  of 
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square-root,  inverse  function,  power,  log,  exponential,  and 
ratio  transformation  of  the  data  were  tested  without  material 
enhancement  of  the  results.  Various  other  univariate 
regression  tests  were  experimented  with,  but  again  with  no 
improvement  of  the  results  noted. 

As  a  final  test,  a  number  of  control  factors  that  affect 
cost  and  schedule  growth  were  introduced.  These  factors 
included  program  stretchout,  a  measure  of  how  government 
attempts  to  meet  budgetary  constraints  by  buying  the  same 
quantity  over  a  longer  period  of  time  or  a  smaller  quantity 
within  the  original  schedule.  Other  control  factors  consisted 
largely  of  acquisition  initiatives,  including  total  package 
procurement,  fixed  price  development,  and  contract  incentives 
in  the  development  and  production  phases.  Program  stretchout 
and  the  acquisition  initiatives  with  the  strongest  affect  on 
cost  growth  were  used  as  additional  independent  variables  in 
a  series  of  multivariate  regressions.  The  results,  while 
still  not  highly  significant,  nevertheless  indicated  that 
ADVANCE  is  useful  as  a  predictor  of  production  and  total 
program  cost  growth. 

B.  CONCLUSIONS 

The  overall  low  significance  levels  and  extremely  small 
explanation  of  variability  in  the  models  establishes  that 
measures  of  technological  complexity  and  extension  are  not 
strong  predictors  of  cost  or  schedule  growth  in  either  the 
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development  or  production  phases  of  tactical  air-launched 
munitions  acquisition.  The  results  obtained  from  both 
development  cost  and  development  schedule  tests  offers  some 
confirmation  that  technological  complexity  is  not  a  direct 
predictor  of  program  outcomes  in  the  development  stage.  The 
use  of  control  factors  as  additional  predictors  of  program 
outcomes  provided  some  improvement  in  the  results.  This 
indicates  that  technology  measures  alone  are  not  strong  or 
dominant  explainers  of  cost  or  schedule  growth,  but  when 
obscuring  factors  are  controlled  for,  technological  extension 
(as  measured  by  ADVANCE)  is  apparently  related  to  production 
and  total  program  cost  growth. 

The  results  have  some  limitations.  The  15  tactical  air- 
launched  munitions  systems  were  a  small  sample.  In  this  small 
sample,  three  distinct  missile  groups  were  merged  together. 
Data  collection  for  technology  variables  was,  to  some  extent, 
confined  to  those  measures  common  to  each  of  the  three  groups. 
This  suggests  that  improved  technology  describing  measures 
could  be  obtained  for  a  more  homogenous  sample  of  missiles. 
It  is  possible,  as  well,  that  cost  and  schedule  growth 
measures  are  not  fully  comparable  between  the  three  m:ssile 
groups.  This  indicates  that  improved  results  may  be  achieved 
through  the  use  of  a  larger  and  more  homogenous  sample  of 
systems. 
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APPENDIX  A 


MISSILE  DESIGNATION  SYSTEM 

A  chart  describing  the  Department  of  Defense  (DOD)  system 
for  identifying  missile  programs  is  provided  below. 
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APPENDIX  B 


PROPERTIES.  CHARACTERISTICS.  AND  PERFORMANCE  MEASURES 
DESCRIBING  TACTICAL  AIR-LAUNCHED  MUNITIONS  TECHNOLOGY 


Acquisition  (Seeker  Range) 

Airframe  Structure 
Attack  Azimuth  Angle 
Aerodynamic  Performance 

-  Boost 

Chaff  Defeating  Capability 

-  Digital  Signal  Processing  Capability 
Electronic  Counter-Measures  Defeating  Capability 
Flare  Defeating  Capability 

Format  Target  Capability 
Fuzing  Success 
Fuzing  System  Performance 
Guidance  Success 

Imaging  (Target/Background  Reflections) 

Inertial  Mid-Course  Correction 
Inertial  Reference 
Kill  Probability  (Pk) 

Launch  Acceptability  Regions  (LARS) 

Launch  Platform  Speed 
Launch  Success 
Lethality 

Look-Down,  Shoot-Down  Capability 
Mid-Course  Guidance  Performance 
Missile  Message  Accuracy 
Missile  Altitude  Capability 
Missile  Range  Capability 
Missile  Speed  Capability 
Multiple  Target  Capability 


No-Escapes  Zones 
Propulsion  Management 
Propulsion  System  Performance 
Sensor  Performance 
Safe  Separation 
Seeker  Mode 
Stability  and  Control 
Software  Assessment 

Software  Validation  and  Verification 

Terminal  Guidance  System  Redundancy 

Terminal  Guidance  System  Reliability 

Warhead  Performance 

Weapon  Effectiveness 

Weapon  Reliability 

Weather 

Primary  Source:  [Refs.  19,20] 
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APPENDIX  C 


YEAR  OF  TECHNOLOGY  REGRESSION  ANALYSIS 


Version  A 

The  regression  equation  is: 

FSDSD  =  65.1+0.38  ALT2  +3.18  TGS2  +4.90  ECCM  +0.87  IMCC 

+2.38  FUZE 


Predictor 

Coef 

Stdev 

t-ratio 

p-val 

Constant 

65.077 

4.064 

16.01 

0.000 

ALT2 

0.377 

1.733 

0.22 

0.833 

TGS2 

3 . 180 

1.963 

1.62 

0.140 

ECCM 

4.896 

5.276 

0.93 

0.378 

IMCC 

0.866 

5.885 

0.15 

0.886 

FUZE 

2.381 

4.393 

0.54 

0.601 

s  =  5.972 

R-sq 

=  51.3% 

R-sq ( ad j)  = 

24.2% 

Analysis  of 

Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

5 

337.90 

67.58 

1.89 

Error 

9 

321.03 

35.67 

Total 

14 

658.93 

SOURCE 

DF 

SEQ  SS 

ALT2 

1 

107.36 

TGS2 

1 

153.69 

ECCM 

1 

65.91 

IMCC 

1 

0.46 

FUZE 

1 

10.48 
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Version  B 


The  regression  equation  is: 


FSDSD  =  69 

.1  +  3.17 

ALT2  +4.96 

ECCM  +  20. 

0  SPD3 

Predictor 

Coef 

Stdev 

t-ratio 

p-val 

Constant 

69.066 

2.180 

31.68 

0.000 

ALT2 

3.169 

1.546 

2.05 

0.065 

ECCM 

4.964 

3.580 

1.39 

0.193 

SPD3 

20.017 

6.510 

3.07 

0.011 

S  =  4.581 

R-sq 

=  65.0% 

R-sq ( ad j)  = 

=  55.4% 

Analysis  of 

Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

3 

428.07 

142.69 

6.80 

Error 

11 

230.87 

20.99 

Total 

14 

658.93 

SOURCE 

DF 

SEQ  SS 

ALT  2 

1 

107.36 

ECCM 

1 

122.29 

SPD3 

1 

198.42 

p-val 

0.007 


Unusual  Observations 

Obs.  ALT2  FSDSD  Fit  Stdev.Fit  Residual  St.Resid 

14  1.00  63.00  72.24  1.76  -9.24  -2.18* 


*  Denotes  an  observation  with  a  large  standardized  residual 


Version  C 


The  regression  equation  is: 


FSDSD  =  68 

.2  +  20.3 

SPD3  +  2.63 

ALT2  +  1.77 

ECCM 

+ 

4.91  IMCC 

+  0.73  TGS2  +3.15  FUZE 

Predictor 

Coef 

Stdev 

t-ratio 

p-val 

Constant 

68.203 

3.472 

19.65 

0.000 

SPD3 

20.301 

8.159 

2.49 

0.038 

ALT2 

2 . 632 

1.651 

1.59 

0.150 

ECCM 

1.767 

4.386 

0.40 

0.698 

IMCC 

4.913 

4.961 

0.99 

0.351 

TGS2 

0.728 

1.848 

0.39 

0.704 

fuze 

3.149 

3.512 

0.90 

0.396 

s  =  4.756 

R-sq 

=  72.5% 

R-sq ( ad j)  = 

51.9% 

Analysis  of 

Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

6 

477.95 

79.66 

3.52 

Error 

8 

180.98 

22.62 

Total 

14 

658.93 

SOURCE 

DF 

SEQ  SS 

SPD3 

1 

34.02 

ALT2 

1 

353.70 

ECCM 

1 

40.34 

IMCC 

1 

28.99 

TGS2 

1 

2.72 

fuze 

1 

18.18 

Unusual  Observations 

Obs.  SPD3  FSDSD  Fit  Stdev.Fit  Residual  St.Resid 

14  0.000  63.00  72.29  2.23  -9.29  -2.21* 

*  Denotes  an  observation  with  a  large  standardized  residual. 
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APPENDIX  D 


NEW/MOD 

TIME 

PROD-U 

PRO 

STRETCH 

C-PRD 

DTC 

MYP 

FPD 

TPP 

I-FSD 

I -PRD 


Source: 


CONTROL  FACTORS  THAT  AFFECT  COST  GROWTH 


New  vs.  modification  programs. 

Time  period:  programs  distinguished  by  year  of 
FSD  start — late  1960s,  the  early  1970s,  the 
late  1970s,  the  1980s. 

Programs  that  started  production  in  1985  or 
before.  Only  programs  with  production  starts 
before  1985  were  considered. 

Prototype  programs  vs.  programs  that  did  not 
use  a  prototype. 

A  measure  of  program  stretchout  equal  to 
production  schedule  growth  divided  by 
production  quantity  growth. 

Programs  that  included  competition  in  their 
acquisition/contracting  strategy. 

Design-to-Cost  contracts. 

Multi-Year  Procurement  contracts. 

Fixed  Price  Development  contracts.  Coded:  0 
without  FDP,  1  with  FPD. 

Total  Package  Procurement  contracts.  Coded:  0 
without  TPP,  1  with  TPP. 

Contracting  incentives  in  the  Full-Scale 
Development  phase.  Coded:  0  without 

incentives,  1  with  incentives. 

Contracting  incentives  in  the  production  phase. 
Coded:  0  without  incentives,  1  with 

incentives. 


[Ref.  ll:App.  A] 


APPENDIX  E 


COST  GROWTH 

OUTCOME 

CONTROL 

MEASURES^ 

SYSTEM 

I-FSD 

FPD 

I-PRD  TPP 

SISEDCH 

AIM-7E 

1 

0 

1 

0 

9.1471 

AIM-7F 

1 

0 

1 

0 

1.1627 

AIM-7M 

1 

0 

1 

0 

1.1667 

AIM-9L 

1 

0 

1 

0 

2.2439 

AIM-9M 

0 

0 

0 

0 

1.0749 

AIM-54A 

0 

1 

0 

0 

1.2245 

AIM-54C 

1 

0 

1 

0 

0.7794 

AGM-88A 

1 

0 

1 

0 

1.5333 

AGM-84A 

1 

0 

1 

0 

3.2105 

AGM-65A/B 

0 

0 

0 

1 

2.1356* 

AGM-65D/G 

1 

0 

1 

0 

1.0974 

AIM-120A 

0 

1 

0 

0 

1.1100 

AGM-114A 

1 

0 

0 

0 

1.2020 

BGM-71A/B 

0 

0 

1 

0 

3.8475 

BGM-71D 

1 

0 

1 

0 

1.0562 

♦STRETCH  FOR  the  AGM-65A/B  Maverick  was  not  provided  in  the 
Tyson  study's  acquisition  program  database.  The  mean  value  of 
STRETCH  was  inserted  to  preclude  a  distortion  of  the  data  for 
use  later  in  the  analysis. 

Source:  [Ref.  llcApp.  A] 


^Refer  to  Appendix  D  for  an  explanation  of  control 
measures. 
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